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We investigate tlie well-posedness and asymptotic self-similarity 
of solutions to a generalized Smoluchowski coagulation equation re- 
cently introduced by Bertoin and Le Gall in the context of continuous- 
state branching theory. In particular, this equation governs the evolu- 
tion of the Levy measure of a critical continuous-state branching pro- 
cess which becomes extinct (i.e., is absorbed at zero) almost surely. 
We show that a nondegenerate scaling limit of the Levy measure (and 
the process) exists if and only if the branching mechanism is regularly 
varying at 0. When the branching mechanism is regularly varying, we 
characterize nondegenerate scaling limits of arbitrary finite-measure 
solutions in terms of generalized Mittag-LefHer series. 

1. Introduction. 

1.1. Overview. Recently Bertoin and Le Gall [3] observed a connection 
between the Smoluchowski coagulation equation and any critical continuous- 
state branching process (hereafter CSBP) that becomes extinct with prob- 
ability one. Our general goal in this paper is to establish criteria for the 
existence of dynamic scaling limits in such branching processes, by extend- 
ing methods that were recently used to analyze coagulation dynamics in the 
classically important 'solvable' cases (i.e., cases reduced to PDEs in terms 
of Laplace transforms). 

Substantial progress has been made in recent years understanding the 
long-time behavior of solutions to solvable Smoluchowski coagulation equa- 
tions. A rich analogy has been developed between dynamic scaling in these 
equations and classical limit theorems in probability, including the central 
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limit theorem, the classification of stable laws and their domains of attrac- 
tion [14, 16], and the Levy-Khintchine representation of infinitely divisible 
laws [2, 18]. 

A new challenge in dealing with the coagulation equations that appear in 
the context of CSBPs is that they typically lack the homogeneity properties 
which were used extensively in earlier scaling analyses. On the other hand, 
use of a Laplace exponent transform leads to the study of a rather simple 
differential equation determined by the branching mechanism of the CSBP. 
Moreover, these branching mechanisms have a special structure — a Levy- 
Khintchine representation formula expressed in terms of a certain measure 
related to family-size distribution. 

To deal with the lack of homogeneity, we will adapt ideas from renor- 
malization- group analysis, studying convergence of rescaled solutions to- 
gether with the rescaled equations they satisfy. Such methods have been 
used to study asymptotic limits in a variety of problems including nonlinear 
parabolic PDE and KAM theory [6, 7]. An important point in this type 
of analysis, and one featured here, is that nontrivial scaling limits, if they 
exist, satisfy a homogeneous limiting equation. We describe these features 
in greater detail below. 

1.2. Continuous-state branching processes. CSBPs arise as continuous- 
size, continuous-time limits of scaled Galton- Watson processes, which model 
the total number in a population of individuals who independently reproduce 
with identical rates and family-size distributions. A CSBP consists of a two- 
parameter random process {t,x) Z{t,x) G [0, oo) (t > 0, x > 0). For 
fixed X, the process t i— > Z{t, x) is Markov with initial value Z{0, x) = x. For 
fixed t, the process x i— >■ Z{t, x) is an increasing process with independent 
and stationary increments. The right-continuous version of this process is a 
Levy process with increasing sample paths. In particular, the process enjoys 
the branching property that Z{t,x + y) has the same distribution as the sum 
of independent copies of Z{t,x) and Z{t,y) for all t ^ 0. 

The structure of the process Z{t, x) has a precise characterization via the 
Lamperti transform. That is, t i— )■ Z{t, x) can be expressed as a subordinated 
Markov process with parent process x + Xt where Xt is a spectrally positive 
Levy process. More specifically, Z{t,x) = x + ^e(j,a;) where the process 
t I— )> Q{t,x) has non-decreasing sample paths and formally solves dtQ = 
X + Xq. In this context, the Laplace exponent of Xt, denoted is called 
the branching mechanism for Z{t, x) and has Levy-Khintchine representation 
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where a G M, /3 ^ 0, and f^^ oo)(^ Ax'^)'K{dx) < oo. The representation (1.1), 
having the property ^'(0^) = 0, assumes no kilhng for the associated CSBP 
(cf. [11]). 

Due to the nature of the Lamperti transform, Z{t, x) satisfies 

(1.2) ^(g-gZ(M)) ^ g-x^(t,9)^ 

where the spatial Laplace exponent <p solves the backward equation 

(1.3) dtifit, q) = -^{ifit, q)), qe (0, oo), t > 0. 

Corresponding to Z{0,x) = x, the initial data takes the form ip{0,q) = q. 
It follows that X I— 7- Z(t, x) is an increasing process with independent and 
stationary increments. As the Laplace exponent of a subordinator, ip has the 
Levy-Khintchine representation 

(1.4) ^{t,q)=btq+ [ (1 - e-'?")z.t(a!x), q>0, 

J(0,oo) 

where bt > and J^q ^-j(l Ax)ft(dx) < oo. The quantities bt and ft represent 
the drift coefficient and the Levy jump measure, respectively. Taking q oo 
in (1.2) one sees that the CSBP becomes extinct in time t with positive 
probability (i.e. ¥[Z{t,x) = 0] > 0) if and only if ip{t, oo) < oo. This means 
that bt = and pt < oo, where 

pt = {ut, 1) = / i^t{dx). 

J{0,od) 

(See Proposition 3.7 for a characterization of branching mechanisms of this 
type.) 

In the present work, we restrict our attention to the class of CSBPs for 
which the branching mechanism \I' has the property 

(1.5) ^''(0+) = a - / X7r{dx) > -oo. 

J[l,oo) 

That is, we assume ^ has the representation 

(1.6) ^{u) = au + Pu'^ + j (e""^ - 1 + n2;)7r((i3;), 

i(0,oo) 

where a G M, /3 ^ 0, and the branching measure TT{dx) verifies 

(1.7) / {x A x^)TT{dx) < oo. 

J(0,oo) 
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As shown in [9, 11], the CSBP associated to (1.6)-(1.7) is conservative in the 
sense that ¥{Z{t, x) < oo) = 1 for ah t > 0. Of primary interest is the case 
of critical branching, which is distinguished by the property K{Z{t, x)) = x, 
and corresponds here to the value a = 0. 

1.3. A generalized Smoluchowski coagulation equation. The connection 
between branching and coagulation may be understood roughly as follows. 
Fix a time horizon T. The Levy jump measure vtidx) has an interpretation 
as the limiting size distribution of clans of individuals descending from a 
common ancestor at time T — t. As t increases, we look further back in 
time for a common ancestor, causing clans to coalesce. The rate at which 
clans merge is governed by the branching measure ^{dx), which roughly 
corresponds to the first-generation family size distribution of an individual. 

As shown in [3], the Levy measure of a critical CSBP which becomes 
extinct almost surely satisfies a generalized type of Smoluchowski coagulation 
equation. This equation belongs to a general class of coagulation models 
that account for the simultaneous merging of k clusters with (possibly time- 
dependent) rate R^. Specifically, the weak form of this equation is 

(18) ^^^ = E^'^^'=(^'^*)' foran/€C([0,oo]). 

k>2 

Here 

(1.!.) .) = 1^^^^^ (/(X, g /(..)) n 

represents the expected change in the moment 

(^,/) = / f{x)u{dx) 

upon merger of k clusters with size distribution v. For the evolution equation 
of the Levy measure of a critical CSBP which becomes extinct almost surely, 
the rate constants i?^ have a particular Poissonian structure expressed in 
terms of the branching mechanism and the total number pt = {uf, !)• Namely 
Rk = RkiPt) where 

ki 7(o,oo) k\ 
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Here, /3 is the diffusion constant appearing in (1.6), and 6k2 is the Kro- 
necker delta function, which is zero for k >3. Combining the relations (1-8) 
and (1.10) gives the coagulation equation 

(1.11) = ^ j:, 4(/, ^t). 

k=2 

In the case of the special branching mechanism ^(u) = ti^, we recover the 
classical Smoluchowski coagulation equation with rate kernel K{x, y) = 2. 
Also, note that a Levy measure solution of (1.11) represents a kind of fun- 
damental solution for the coagulation equation, having the special property 
that as t — 7- the measure xvt{dx) converges weakly to a delta function at 
the origin (see Remark 3.9). 

1.4. Results and organization. 

1.4.1. Characterization of scaling limits for coagulation. Our main re- 
sults relate to long-time scaling limits of measure solutions of the coagula- 
tion equation (1.11) where ^ is a critical branching mechanism for a CSBP 
which becomes extinct almost surely. That is, we investigate the existence 
of dynamic scaling limits of the form 



(1.12) a{t)ut{X{t)"'^dx) i){dx) as t 



oo, 



for functions a, A > and finite measure z>. We show that the existence of 
nondegenerate limits is fundamentally linked to two conditions: 

(i) Regular variation of ^ at zero with index 7 € (1,2]. 

(ii) Regular variation of the mass distribution function yut{dy) at in- 
finity with index 1 — p, where p S (0, 1]. 

First, assuming condition (i) holds, we prove (Theorem 5.1) that scaling 
limits of the form (1.12) exist if and only if condition (ii) holds at some 
initial time t = tQ ^ 0. Since initial data satisfying (ii) are easily constructed, 
condition (i) gives a sufficient condition under which (1.11) admits nontrivial 
scaling solutions. The remarkable fact (Theorem 6.1) is that condition (i) is 
both necessary and sufficient for the scaling limit (1.12) to exist when i/f is 
the fundamental solution (defined in Section 3.2). 

The theorems cited above also provide a precise characterization of the 
limiting measure 0. Specifically, we show that (i) and (1.12) together imply 
that there exist constants cx > and p € (0, 1], the latter given by (ii), such 
that 

(1.13) i>(dx) = (i>, l)F^,p((i>, l)h^^dx), 
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where -F^^p is a generalized Mittag-Leffler probability distribution given by 

where r = (7 — s = p{'y — l), and {r)k denotes the Pochhammer symbol 

{r)k = r{r + l)(r + 2) • • • (r + /c - 1). 

Moreover, the corresponding solution i/t is asymptotically self-similar in the 
sense that for all t > 0, 

(1.15) a{T)urt{\{Ty^dx) t^uifi^o^dx) 

as r — 7- 00. In particular, the limiting function in (1.15) belongs to the family 
of self-similar solutions of (1.11) with homogeneous branching mechanism 
of the form "^(u) = fiu^ , where /3 = (7 — l)~^(z), 1)^"'^. These solutions have 
the form 

ut{dx) = a{t)F^^p{a{typc^^dx), a{t) = [/3(7 - l)t]^, 

which generalizes the one-parameter family obtained in [16] corresponding 
to the classical Smoluchowski equation, with 7 = 2 and c\ = 1. 

1.4.2. Limit theorems for critical CSBPs. Theorems 5.1 and 6.1 estab- 
lish a necessary and sufficient condition for the existence of nondegenerate 
scaling limits of fundamental solutions, namely. Condition (i), above. We 
now describe two rather direct consequences of this fact in terms of scaling 
limits of the corresponding CSBP. 

First, given a CSBP Z{t, x) for which the corresponding Levy measure is 
a fundamental solution of (1.11), we consider scaling limits of the form 

(1.16) \{t)Z{t,a{t)x) ^ Z{x) 

as t ^ 00. That is, we scale by a factor of \{t) the total population at 
time t descended from an initial population of size a{t)x, and we investigate 
the convergence in law of the rescaled process, with parameter x, to a non- 
degenerate Levy process Z. As above, we prove that such a limit exists if 
and only if Condition (i) holds. This is Theorem 7.2. In particular, if (1.16) 
holds, then for each t > 

(1.17) \{T)Z{Tt,a{T)x) ^t'^Z{t^x) 
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as r — )■ oo, and the right-hand side is equal in law to the CSBP with Levy 
measure given by t^-i jl{t'^-~i dx), where fi is the Levy measure of Z. In this 
way, we establish the self-similar form of the limiting CSBP. 

Alternatively, one can consider initial population as fixed, and obtain a 
conditional limit theorem for critical continuous-state branching processes 
conditioned on non-extinction. In the context of discrete-state branching, 
several authors [5, 20, 23] have investigated limits of the form 

(1.18) P(A(t)Zt ^ X I Zt > 0) ^ F{x) 

as t — )■ cx), where Zt is the branching process and F is a nondegenerate 
distribution function on (0, oo). By various techniques (our own being most 
similar to a method of Borovkov [5]), the authors prove that for the special 
scaling function \{t) = ¥'{Zt > 0) a limit of the form (1.18) exists if and only 
if the process Zt has an offspring law corresponding to a regularly varying 
probability generating function. The question of whether the same regular 
variation condition is implied for a general scaling function A(t) — >• was 
left open by Pakes [20]. Theorem 7.1 provides an affirmative answer to the 
continuous-state analog of the question posed by Pakes as an easy corollary 
of Theorem 6.1. 

Also implied by Theorem 7.1 are the conditional limit theorems obtained 
by Kyprianou [12] for critical CSBPs with power-law branching mechanism 
(the so-called a-stable case), and those obtained by Li [15] for critical CSBPs 
with the property \I'"(0^) < oo. In all cases above, including the discrete 
cases previously mentioned, limiting distributions are characterized by rela- 
tions of the form (1.14). 

Let us note that, by comparison, non-critical CSBPs admit scaling limits 
of a simpler form. Indeed, a well-known result of Grey [9] states that for any 
supercritical CSBP with ^''(0"'") > — oo, and for any critical or subcritical 
CSBP with ^''(oo) < oo (in the latter case, the CSBP remains positive 
almost surely - see Proposition 3.7(i)), there exists a scaling limit of the 
form (1.16), where a{t) = 1 and ip{t,X{t)) = const., with if solving (1.3). 
On the other hand, it follows directly from the work of Lambert [13] that 
any subcritical CSBP which becomes extinct almost surely admits a limit 
of the form (1.16) with scaling functions given by a{t) = l/<p{t,oo) and 
A(t) = 1. In contrast with Theorem 7.2, only one nontrivial scaling function 
is needed in each of the cases above. 

1.4.3. Well-posedness. For the sake of completeness we also give an ac- 
count of well-posedness for the coagulation equation. That is, we establish 
the existence and uniqueness of weak solutions of (1.11) when ^ is a criti- 
cal branching mechanism and the initial data is a finite measure (Corollary 
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3.12). Here, we essentially tie together the ideas of Bertoin and LeGall [3], 
Norris [19], and Menon and Pego [16] with a few new proofs and observa- 
tions. In particular, we provide a simple and direct account of well-posedness 
for the evolution of the Levy measure vt in (1.4) (Proposition 3.7). The point 
is that equation (1.3) preserves the property that (p{t, •) has a completely 
monotone derivative. For an initial cluster size distribution given by a finite 
measure, the latter property amounts to a well-posedness result for Smolu- 
chowski dynamics. 

1.4.4. Outline of the paper. We now give a brief outline of the paper. Sec- 
tion 2 delineates some basic notation and definitions. Section 3 is dedicated 
to well-posedness results. In Section 4, we derive the family of self-similar 
solutions to (1.11) associated with generalized Mittag-Leffler laws. Section 
5 is dedicated to a study of scaling limits of the form (1.12) in the case of a 
regularly varying branching mechanism ^ . In Section 6, we consider scaling 
limits of fundamental solutions. Finally, in Section 7, we reformulate our 
scaling results in terms of limit theorems for CSBPs. 

2. Preliminaries. We begin with some notation that will be repeatedly 
used throughout this paper. Let E be the open interval (0, oo), and E denote 
the extended interval [0, oo]. We use C{E) to denote the space of continuous 
functions f : E ^ M, equipped with the L°°-norm. 

Three spaces of measures that arise often in our context are: 

• The space consisting of positive Radon measures on E equipped 
with the vague topology. We recall that if /i, /^i, /i2, • • • are measures in 

then fin converges vaguely to ^ as n — > oo (denoted by fin — > fi) 
if {fin,<P) — >■ (a*, for all (j) € Cc{E). Here Cc{E) denotes the space of 
continuous functions on E with compact support, and {fi, f) denotes 
the integral of / with respect to the measure fi. 

• The space Mf, consisting of finite positive measures on E, equipped 
with the weak topology. That is, if fi, fii, fi2, ■ ■ ■ sue measures in Mf, 
then we say /U„ converges weakly to as n ^ oo (denoted by fj-n — > Z^) 
if {nn-,4') ~^ {l^i4>) for all £ Cb{E). Here Cb(E) denotes the space of 
bounded continuous functions on E. 

• The space Miax, consisting of the set of measures /i G such that 




(0,oo) 



2.1. Branching mechanisms and Bernstein functions. 
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Definition 2.1. We say a function ^ : — )• M is a branching mechanism 
if it admits the representation 

(2.1) ^{u) = Qu + /3nV / (e"""^ - 1 + ux) Tr{dx), 

Je 

where q € M, /3 ^ and vr € M.+ with f^{x A x^)7r(dx) < oo (equivalently, 
XTT € A^iAx)- The branching mechanism is called critical, suhcritical, or 
supercritical according to the conditions d = 0, a > 0, or a < 0, respectively. 

Definition 2.2. We say that / G C°°{E) is a Bernstein function if 
/ ^ and (-l)fc/('=+i) ^ for all integers ^ 0. 

In other words, / is a Bernstein function if / is non-negative and /' is 
completely monotone. It is well known (see for instance [22]) that a function 
is Bernstein if and only if it admits the representation 

(2.2) f^q)=a + hq+ [ {1 - e-"^) ^,{dx), 

Je 

where a,b ^ and /i € Miax- Note that / is strictly positive if and only if 
(a, b, /i) 7^ (0, 0, 0). On the other hand, a function ^ : E E belongs to the 
set of critical or subcritical branching mechanisms if and only if ^'(0"'^) = 
and ^' is a positive Bernstein function. The following lemma, for which we 
have found no obvious reference, establishes a deeper relation between set 
of critical or subcritical branching mechanisms and Bernstein functions. 

Lemma 2.3. Assume ^: E ^ E is a critical or subcritical branching 
mechanism. Then, the inverse function is a Bernstein function. 

Proof. Let / = and g = 1/^'. Note g is completely monotone, since 
X I— >■ 1/x is completely monotone and ^' is a positive Bernstein function, as 
observed above. Since / is positive and f' = gof,it directly follows that /' 
is completely monotone, from [18, Lemma 5.5]. □ 

3. Well-posedness for Smoluchowski dynamics. In this section we 
define a notion of weak solution for the generalized Smoluchowski equa- 
tion (1.11). As we will show, the question of existence of weak solutions 
amounts to a study of (1.3). Several estimates appearing in Sections 3.1 and 
3.3 have either been sketched in [3] from a probabilistic point of view, or are 
straightforward extensions of the well-posedness theory in [16]. The origi- 
nality of our treatment lies mainly in Lemma 2.3 and its use in the proof 
of Proposition 3.7. The remaining estimates have been simplified by various 
degrees and organized for convenience of the reader. 
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3.1. Weak solutions. In this section, we consider a critical branching 
mechanism ^ having the representation (2.1) with a = 0. Following the 
approach in [16, 19], we associate to each finite, positive measure u £ A4p 
the continuous linear functional L{i') : C{E) M, defined by 

(3.1) (L(z.), /) = i?,((z., i))4(/, z.) = f; ^) 



k>2 



k=2 



kl 



where Ik and are defined by (1.9) and (1.10), respectively. To verify con- 
tinuity of L(i/), we observe |/fc(/,z^)| < (A;-|-l)||/||^^;gj. Thus for m = {u, 1), 
equations (2.1) and (3.1) give 



\{L{J^)J)\ 



k=2 



[mxj 



■ TT{dx] 



C{E) 



oo h k 

m X 



oo h k 

m X 



(fc- 1)! ^ ^ kl 

.k=2 ^ ' k=2 



n{dx) 



^MII/llc{E)' 



where K(m) = 3/3m^ + 2m'if'{m) — ^'(m) < oo, establishing continuity of 
L(i^). Observe for future use, that 



(3.2) 



K{m) 



-i3u + 2u'^"{u) + ^'{u) 



du. 



Since ^' , ^" ^ 0, the function m ^ K[m) is positive and increasing. 
With this, the natural notion of weak solutions to (1.11) is as follows. 

Definition 3.1. We say that a weakly measurable function v. E ^ Mp 
is a weak solution of (1.11) if 



(3.3) 



f) = {u„f)+ [\L{u^)J)dT 

J s 



for all t, s > and for all / G C{E). If, additionally, there exists uq £ Mp 
such that vt converges weakly to fo as t — ?• 0, then we say v. [0, oo) Mp 
is a weak solution of (1.11) with initial data vq. 



To any function v: E ^ A^iazi we associate the function 



(3.4) 



Lp{t,q) 



(1 



vt{dx). 



LIMIT THEOREMS FOR CRITICAL CSBPS 



11 



Our next result shows that weak solutions to (1.11) are characterized by (1.3) 
for the associated function ip. 

Theorem 3.2. Let ^ : E ^ E he a critical branching mechanism. As- 
sume u: E ^ A4f and that (p is related to v by (3.4)- Then, u is a weak 
solution of (1.11) if and only if if solves (1-3). 

Proof. Let v and (p be as described. First, we claim that p satisfies (1.3) 
if and only if (3.3) holds for the family of test functions fq{x) 1 — e~'^^ , 
< q ^ oo. Note, carefully, that we include the function /oo = 1 in this 
family. Indeed, since ft G A^^?, we have 

<f{t, q) ip{t, oo) = (ft, 1) < oo 

as g ^ oo, so that if (1.3) holds for < g < oo, it also holds for q = oo. 
Now, for any < q ^ oo, we have 



hifg 



1=1 



dv' 



-qxi 



1=1 



i=l 



-qXi 



dul 



{uu 1)' - {{yu 1) - <^(t, q)T - kWt, i)'^~ V(t, q) 



Consequently, using the Taylor expansion for ^ about m = {i^t, 1), we have 



{L{ut),f,) = J2' 



k=0 



- {m - ip(t,q))^ - km'' ^ip(t, 



^{0+)-^{^{t,q))+p{t,q)^'{0 



-^o^{t,q). 



Note that the terms corresponding to /c = 0, 1 in the sum above are clearly 
0. The last equality follows because ^(O"*") = ^'(0^) = 0, by assumption. 
Therefore (3.3) holds for f = fqii and only if 



^{t,q) - V{s,q) 



^((^(r, q))dT 



for all s,t > 0. This proves the claim. 

In particular, if f is a weak solution of (1.11), then ip solves (1.3). On 
the other hand, if p solves (1.3), then (3.3) holds for all test functions fq, 
< g ^ oo. This family of test functions spans a dense subset of the metric 
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space C{E). Now, given / G C{E) and e > 0, choose g G span{/q : < (7 ^ 
00} such that 11/ — g\\ < e. By Hnearity, (3.3) holds for the test function g. 
Therefore, assuming for definiteness that t > s, we have 



if - 9: T^t) - if - 9, l^s) - I if -9, L{Vr)) dT 

J s 

^ 11/ - 9\\c(E) {{^t, 1) + {^s, 1) + ^ K{{Vr, 1)) dT 

^ e[2{us,l) + {t - s)K{{us,m, 

where the function K is given by (3.2). Taking e — )■ shows that (3.3) holds 
for all / € C{E). This completes the proof. □ 

Remark 3.3. Bertoin and LeGall [3] propose a weaker form of Smolu- 
chowski's equation that requires only ut S AIiaxj not Mp-, but which still 
transforms to (1.3). In particular, they show that if ^''(oo) = 00 (see Propo- 
sition 3.7, below), then the Levy measure of the associated CSBP verifies 
this weak form for the special test functions fq, < q < 00. However, 
there appear to be no obvious estimates available to deal with a general test 
function f e C(E). 

3.2. Fundamental solutions. For any weak solution u: E —?■ Aip the 
generalized Smoluchowski equation (1.11), the solution ip{t,q) of (1.3) has 
a finite limit as t whether or not z^^ has a weak limit as t — > 0. Indeed, 
if (/^(i, go) — ^ 00 as i — )• for some qo > 0, then, by a translation invariance 
of solutions, one shows that for any q > qo there exists tq > such that 
ip{t, q) = ip{t — tq, go) — ^ 00 as t ^ tq, which contradicts utq € A^f- 

It follows that if has the convenient representation 

(3.5) ^{t,q) = ^{t,ip{Q,q)), 

where 99(0, q) '= 9^(0^, q) and where <I> solves the initial value problem 



(3.6) 



dMt,q) = -^{Ht.(i)), Q^E 

«>(0,g) = q. 



The functions = ^{t^ ■) have the semigroup property ^t+s = for 
t, s > 0. Because of the composition structure (3.5), we make the following 
definition. 
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Definition 3.4. Assume ^ : E E is a critical branching mechanism. 
We say that a function fj,: E ^ Mp is the fundamental solution of the 
generahzed Smoluchowski equation (1.11) if the function 

(3.7) ^t,q)= [ {l-e-ni^t{dx) 

Je 

solves the initial value problem (3.6), where $(0, g) = ^{0^,q). 

The fundamental solution relates solutions of the generalized Smoluchow- 
ski equation to their initial data via solutions of a linear problem; see Re- 
mark 3.10 below for details. But first we establish necessary and sufficient 
criteria for the existence of a fundamental solution, and develop the basis 
for our discussion of well-posedness theory for weak solutions with initial 
data. For this purpose, we adopt the following terminology from [3]. 

Definition 3.5. We say that a branching mechanism ^ : E satis- 
fies Condition E provided ^(oo) = oo and 

roc. ^ 

(3.8) / ^ , , du < oo for some a > 0. 

Remark 3.6. It is well-known that Condition E gives a necessary and 
sufficient condition under which solutions to (1.3) have finite-time blow-up, 
backward in time. 

Proposition 3.7. Let ^ be the unique solution of the initial value prob- 
lem (3.6), where ^ : E IB. is any branching mechanism of the form (2.1). 
Then, for each fixed t ^ 0, the map ^{t,-)E — > E is a Bernstein function. 
More precisely, 

(3.9) $(t, q) =btq+ [ (1 - e-'^')fit{dx) 

Je 

for some bt ^ and fit £ Miax- Furthermore, the following properties hold. 

(i) bf = for some (equivalently all) t > if and only if ^''(oo) = oo. 
(ii) bt = and fit ^ -Mp for some (equivalently all) t > if and only if ^ 
satisfies Condition E. 

Remark 3.8. While the facts above can be infered from CSBP theory, 
we summarize them here for convenience of the reader, and give a proof 
independent of the latter theory. In particular, we recognize equation (3.9) 
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as the Levy-Khintchine formula for the Laplace exponent of a CSBP with 
branching mechanism ^, as sketched in Section 1.2. In this context, property 
(ii) states that a CSBP becomes extinct by time t with positive probability 
oo) < oo) if and only if Condition E holds. For critical CSBPs, this 
is the case if and only if the process becomes extinct almost surely. Thus, 
property (ii) establishes a one-to-one correspondence between fundamental 
solutions of (1.11) and Levy measures for critical CSBPs that become extinct 
almost surely. 

Proof. Our proof is based on the implicit Euler method. First we will 
show that each iteration of the implicit Euler scheme for (3.6) yields a Bern- 
stein function. Then, since the set of Bernstein functions is closed under 
composition and pointwise limits [22, pp. 20-21], convergence of the implicit 
Euler scheme implies that $(t, •) is Bernstein. 

By assumption, ^ has the representation (2.1). Since ^' is increasing 
and ^''(0^) = a € M, it follows that ^ is Lipshitz on bounded intervals. 
Hence (3.6) has a unique solution. Furthermore, the solution remains positive 
for all time since the equation is autonomous and \I'(0"'") = 0. Also, since 
dt^ = -^'($) ^ -a^, we obtain, for all t, g ^ 0, the bound 

(3.10) ^ $(t,g) ^ ge""*. 

For fixed t > and € N, let /i = t/N and consider the iteration scheme 

(3.11) K+i{q) = Kiq)-h^{^n+i{q)), n = 0,1, . . . , N - 1. 
Note that for sufficiently large, the function Fn : E E defined by 

(3.12) Fn{u) = u + ^^{u) = u + h^{u) 

is a bijection, since Fj^{u) = 1 + h^'{u) ^ 1 + a/i > 0. By consequence, 
^n+iio) = F^^{^n{(l)) is well-defined and positive for all g > and n = 
0, 1, . . . , A^ — 1. Since ^ is locally smooth on E and we have the bound (3.10), 
the proof of the pointwise convergence ^n{(1) ^{t-, (?) as A^ — oo for each 
g > is standard and we omit it. 

Observe now that F^ is a branching mechanism since it has a represen- 
tation of the form (2.1). Hence by Lemma 2.3, F^^ is a Bernstein function, 
provided A^ is sufficiently large. Since the set of Bernstein functions is closed 
under composition, and $0(9) = 9 is a Bernstein function, it follows is a 
Bernstein function for each n = 0, . . . , A^. Finally, the pointwise convergence 
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$7V ^(t, •) as — )• oo implies that <I>(i, •) is a Bernstein function, by [22, 
Cor. 3.8]. The representation (2.2) then gives 

(3.13) $(t, q)=at + hq + / (1 - e-^^)^t(dx), 

Je 

for some at, h and A^iax- Note that (3.10) imphes at = $(t, 0^) = 
for all t ^ 0. 

Next we establish (i). Observe that bt = dq^{t, oo), and that the relation 

(3.14) dg<^{t,q) = e-/o 

is an easy consequence of (1.3). If ^''(oo) < oo, then since ^' and <I>(s, •) are 
increasing, for any i > we find 

bt = e-/o *'(*{^'°°))'^^ ^ e-**'(~) > 0. 

Conversely, suppose 6j > for some t > 0, then (3.13) implies $(t, oo) = oo 
and hence $t = $(t, •) is a surjection onto E. Since $f = $s o $f_s for 
< s < t, also a surjection and hence $(s,cxd) = cx). Thus bt = 

g-i* (oo) y Q jjence, ^''(oo) < oo. This completes the proof of (i). 

Finally, let us show that (ii) holds. First suppose bt = and j-Lt ^ for 
some t > 0. We claim Condition E holds. From (3.9) we have 

(3.15) $(t, oo)= / fit{dx) < oo. 

Je 

Assume for the sake of contradiction that ^'(oo) < oo. Then ^''(oo) ^ 0, 
and we have by (2.1), /3 = and a = ^''(0^) ^ — / X7r{dx). In that case, 
^'(it) < /g(e~"^' — l)'K{dx) < for all u £ E, and <!>(•, g) is increasing. 
Hence $(t, g) ^ g — )■ oo as q — )■ oo, which contradicts (3.15). This shows 
^(oo) = oo. 

Now, assume (3.8) fails. As remarked above, failure of this condition en- 
sures that all solutions of (1.3) with finite initial data remain finite backward 
in time. In particular, by uniqueness and positivity of solutions of (3.6), we 
have that for all q > 0, 

q = $(0, q) = $(-t, ^{t, q)) ^ #(-t, $(i, oo)), 

which is finite and independent of q. Note that we used monotonicity of <& 
in q for the inequality. This is a contradiction. Hence, Condition E holds. 
Conversely, assume Condition E holds, and let 



q, = mi{qeE:^{q)>0} 
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denote the largest equilibrium solution of (1.3). Then, for any q > there 
exists tg < such that E 3 $(t, (?) — )• oo as i — )■ i^. We define the special 
solution 

^,it) = ^{t-tg,q), 

which is independent of g > and has the property E 3 ^^,{1) ^ oo as 
t 0+. Since ^{0,q) < <I>*(0~'") = oo, we deduce, by uniqueness of solutions 
of (1.3), that <^(i, q) < ^*{t) for all t,q > 0. Therefore, taking g — > cx), shows 
$(t, oo) < oo. That is, h = and ^ Mp, for ah t > 0. □ 

Remark 3.9. Note that the Bernstein functions $(i, •) converge point- 
wise to the function ^{0, q) = q as t ^ 0. It follows that 

dg^{t,q)=bt+ I e-i'^xfitidx) 
Je 

converges pointwise to (9g^'(0, g) = 1 = /[q oo) as t — > (see, for 

instance, [22, p. 21]). Therefore, by the continuity theorem (cf. [8, XIII. 1]), 
the measures K^*\dx) = bt5o{dx) + x^t{dx) converge vaguely to the measure 
5Q{dx) in the space of positive Radon measures on [0,oo). In particular, 
if ^ is a critical branching mechanism satisfying Condition E, then the 
mass measure, xfit{dx), converges weakly to a delta mass at zero as t — > 
0. Moreover, the total mass at time t, given by dq^{t,0^), is conserved 
by (3.14). 

Remark 3.10. Formula (3.5) has a standard probabilistic interpreta- 
tion: For fixed t, the Levy process with Levy measure is subordinated 
to the Levy process with Levy measure vq by the directing process Z{t, •) 
with Levy measure fif In terms of generators, this corresponds however to 
a deterministic formula that expresses the weak solution ft of the nonlin- 
ear Smoluchowski equation in terms of the fundamental solution and the 
solution of a linear problem. Namely, 

(3.16) vt{dx) = / Qs{dx) fit{ds), 

Je 

where Qs is the kernel for the convolution semigroup e^^ (a Levy diffusion) 
with generator determined by VQ{dx). That is, for all smooth / € Cc(K) we 
have 

(3.17) e'^f{x)= [ f{x + y)Qs{dy), Af{x)= [ + y) - f {x))u^[dy) . 

Jr Je 

These formulas are straightforward to establish by Laplace transform, after 
multiplying (3.16) by 1 — e""^^ and s~^{e''^ — I)f by e*^^ and integrating on 
M. (Note that Qs(dx) retains an atom at with mass e~^^'^'^'^\) 
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3.3. Weak solutions with initial data. In this section we establish the 
existence and uniqueness of weak solutions of (1.11) with initial data fo E 
Mf. 

Lemma 3.11. Assume ^ : E —^M is a branching mechanism, and let uq G 
Mi/sx- Then, there exists a unique vaguely continuous map v: [0, oo) 
such that (p, defined by (3.4), is a solution of (1-3) with initial data 

(3.18) ^Q{q) = / (1 - e-'i^yoidx) 

J E 

Furthermore, if i^o £ A4f, then vt £ Mp for all t > 0, and v: [0, oo) Mp 
is weakly continuous. In this case, we have for t > 

(3.19) ^ = -^((.,,1)). 

Proof. Note that (3.5) represents the unique solution of (1.3) with ini- 
tial data 93(0, q) = ipo{q). By Proposition 3.7, the map <^>(t, •) is a Bernstein 
function for all t ^ 0. Also, ipo is a Bernstein function as it admits a repre- 
sentation of the form (2.2). Therefore the composite function, given by (3.5), 
is a Bernstein function for all t^O. Furthermore, we have <p{t,0^) = and 

dgip{t, 00) = lim dg^{t,ipo{q)) ■ lim i^o(g). 

By assumption, the latter limit vanishes, and since dg^{t, •) is decreasing, 
the former limit is finite. Hence, dq(p{t, 00) = and the representation (2.2) 
for (f reduces to 

(3.20) ^it,q)= / Mdx), 

Je X 

for some fit S with jit £ Mi^x- Note that the measure /it is deter- 
mined uniquely by its Laplace transform dqif{t,q). Further, dgip{t,q) is con- 
tinuous in t, since dtdgip{t,q) = 'ii'{ip{t,q))'if{ip{t,q)). Therefore, viewing fit 
as a measure on [0, 00) (which assigns measure zero to the point {0}), it fol- 
lows from the continuity theorem (cf. [8, XIII.l]) that the map fi: [0, 00) — > 
A^+([0, 00)) is vaguely continuous, where A1+([0, 00)) is the space of Radon 
measures on [0,oo). In particular, for all / G Cc{E) C Cc([0,oo)), we have 
(/, fis) if, fJ-t) as s — > t. That is, the map fi : [0, 00) M.+ is vaguely con- 
tinuous. Hence, the map t ^ vt'= x~^ fit G M.+ is also vaguely continuous 
(since fig — > f^t implies (p ■ fig (/) ■ fit ioi any (f> G Cc{E)). This establishes 
the first part of the lemma. Finally, observe 

(ut,!) = lim ip{t,q) = lim <^{t,^Q{q)) = ^>(t, (z^o, !))■ 
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Thus if & M.F, equation (3.19) follows from (3.6) by taking q = {uq, 1). 
Since (3.19) implies t {i^tA) is continuous on [0,oo), we conclude that 
u: [0, oo) Mf weakly continuous (see, for instance, [1, Theorem 30.8]). 

□ 

Corollary 3.12. Assume E E is a critical branching mecha- 
nism, and letuQ^Mp. Then, there exists a unique weak solution of (1.11) 
with initial data vq. 

Proof. First, by Lemma 3.11, there exists a weakly continuous map 
u: [0,oo) — )• M.F such that ip, defined by (3.4), satisfies (1.3) with initial 
data (3.18). In particular, vt ^ Mf converges weakly to vq as t ^ 0. By 
Theorem 3.2, u restricted to E verifies (3.3). Hence, by definition, v is weak 
solution of (1.11) with initial data vq. Uniqueness of the solution follows 
from Lemma 3.11. □ 

4. Self-similarity and Generalized Mittag-Leffler functions. Re- 
call from [16], that the classical Smoluchowski equation, which corresponds 
here to the special branching mechanism '^{u) = n^, admits a one-parameter 
family of self-similar solutions of the form 

vt{dx) = t~^Fp{t~~p dx), t > 0, < p ^ 1, 

where Fp is given by the classical Mittag-Leffler distribution function, sat- 
isfying 



We now discuss the existence of self-similar solutions for homogeneous 
branching mechanisms of the form 

(4.2) ^{u) = /3u^, 1< 7 ^ 2, /3 > 0. 
As in [16], we look for self-similar solutions of the form 

(4.3) ut{dx) = a{t)F{X{tyUx), 

where -F is a probability distribution and a, A > are differentiable. In this 
case, the function (p, defined by (3.4), takes the form 

(4.4) ip{t, q) = aitMqXit)), pis) = / (1 - e-^^)F(dx). 

Je 
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Furthermore, by Theorem 3.2, ip satisfies the equation 

(4.5) dt^{t,q) = -P^{t,qy 

for aU ^ g ^ oo, where tp{t^oo) == f^utidx). By (4.4), ip{t, oo) = a{t). 
Hence, up to the normahzation a(0"^) = oo, (4.5) gives 

(4.6) a(t) = [(7-l)/3t]T^. 
Now, given (4.4), we rewrite (4.5) as 

^^{qXit))+qait)'-''X'{tW{qXit)) = -/3^(gA(t))^ 
In terms of the variable s = qX{t), separation of variables yields 

(7-i)tV(t) _ Hs)-^{sy _ 1 

X{t) s^'is) p' 

where we label the separation constant as 1/p for convenience. The constant 
/? disappears thanks to (4.6). Solving for the general solution in each case, 
we obtain 



(4.7) X{t) = cit^^, <p{s) 



1 



1 + C2S-^(T-1) 



1 

7-1 



where ci, C2 > are arbitrary constants. Taking into account (4.4), we have 
<f>{0~^) = 0. Therefore, p > 0. Furthermore, the fact that (p is a Bernstein 
function implies that < p ^ 1, otherwise (p" takes positive values near 
s = 0. We obtain the following proposition. 



Proposition 4.1. Assume ^ is given by (4.2). Then (1.11) admits a 
one-parameter family of self-similar solutions, indexed by p & (0,1], of the 
form 

(4.8) /if'^'^(dx) = a{t)F^^p{a{typdx), a{t) = [(7 - l)/3t]^, 
where F^^p is a probability measure determined by the relation 

(4.9) /" (1 - e-^^)F^,,(dx) = 



1 



1 + q-pi-y-'^) 



7-1 
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More precisely, the function 



(4.10) 



.^'^'P{t,q)= / (1 - e-''^)/if''^'''((ix) 



solves (4.5) with initial data ip^'"''P{0,q) = qf. In particular, /^f''^'"'^ is the 
fundamental solution of (1.11). That is, fi^''^'^ is the Levy measure for 
Zp^{t,x), the continuous- state branching process with branching mechanism 
(4.2). 

Proof. We set ci = [(7 - l)/3]^^(^ and C2 = 1 in (4.7), so that (4.4) 
takes the form 



(4.11) 



if' 



1 



1 

7-1 



.(7-l)/3t + g-p(')'-i)_ 

By construction, this function solves (4.5) and has initial data (/?^''^'''(0, q) = 
qf, which is a Bernstein function. Hence, by Proposition 3.7, formula (3.5), 
and Theorem 3.2, Lpl^^'^'P has the representation in (4.10) where /^f'^''' solves 
(1.11). The remaining statements regarding the case /f = 1 follow easily from 
definitions - see Sections 1.2 and 3.2. □ 

Finally, we show that the distribution function for F^^p has a generalized 
Mittag-Leffler structure analogous to (4.1). 

Lemma 4.2. Suppose F is a probability measure on E such that for some 
fixed r, s > 0, 



(4.12) 



(1 



)F{dx) 



1 



1 + 



for all g > 0. Then the distribution function of F takes the form 
(4.13) 



F{x) = f2^^ (-1)'^+V'= 



k=l 



kl r(sA; + l) ' 



where (r)^ denotes the Pochhammer symbol, or "rising factorial" function 
{r)k = r{r + l)(r + 2) • • • (r + A: - 1). 

Remark 4.3. A study of generalized Mittag-Leffler distribution func- 
tions of the form (4.13) is given by Prabhakar [21]. In particular, we may 
define, as in [21], the family of generalized Mittag-Leffler functions 



(4.14) 



k=0 



{ak + p) ' 



a,l3,p> 0, 
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in which case (4.13) has the particular form 

F{x) = l-ElA-x 



Proof. By series expansion of (1 — x) ^ aX x = 0, one easily computes 
that 



1 



l + q- 

for \q\ > 1. Next, note that 



r(r + 1) . . . (r + - 1) ^ 

k=0 



q = -—^ — - / e '^'"x"'' ^ dx, 



risk + 1) 70 

for k ^ 1. Since 

1 - / e-'^^F(dx) = ^(-1)^-^^ = 1-5: 

fc=o ■ fc=i 
we conclude, formally, that 

(4.15) F{dx) = y ^ • x^'^-i dx. 

fc=l V I ; 

Indeed, the previous series converges for all a; > 0, has a (probability) dis- 
tribution function given by (4.13), and satisfies (4.12) for all Re(g) > 1. It 
follows by the identity theorem, that (4.12) holds for all g > 0, since both 
the left and right hand sides of (4.12) are analytic for Re(g) > 0. □ 

5. Scaling limits with regularly varying ^ . Proposition 4.1 estab- 
lishes the existence of a family of self-similar solutions of (1.11) with power- 
law branching mechanisms ^'(m) = /Sn"^, 1 < 7 ^ 2. These solutions have a 
scaling invariance given by 

^l^^^P{dx) = s^/if,'^'''(s^^dx), 

for any s > 0. While self-similarity arises in this case due to homogeneity of 
^, we will show that branching mechanisms with an asymptotic power-law 
structure admit solutions which are asymptotically self-similar. 

Recall that a function / > is said to be regularly varying at zero (re- 
spectively, infinity) with index p G M if 

fit) 

as t ^ (respectively, t — > 00) for all x > 0. If p = 0, then / is said to be 
slowly varying. 
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Theorem 5.1. Let v. E ^ Mp be a weak solution of equation (1.11) 
where is a critical branching mechanism which is regularly varying at zero 
with index 7 € (1,2]. 

(i) Suppose there exists a nonzero u £ Aip and functions a,A > such 
that 

(5.1) a{t)h't{\{t)~^ dx) ^ i'{dx) as t — > 00. 
Then, there exists p € (0, 1] such that for all t > 0, 

/•X 

(5.2) / yh't{dy) ^ x^^PL{t,x) as x — > cxd, 



where L{t, •) is slowly varying at infinity. Furthermore, there exists 
c\ > 0, given by (5.15), such that 

(5.3) i){dx) = (i>, 1) F^,p((i>, l)~pc^^dx). 

Here F^^p is the generalized Mittag-Leffler distribution defined by (4.9). 
Moreover, for all t > 0, 

(5.4) a{s)vst{\{s) dx) t^-'i v{tp^^-i) dx) as s ^ 00, 

and the limit in (5.4) is a self-similar solution of (1.11) with branching 
mechanism 

(5.5) ^{u) = (3u\ /3 = ^ ' \ . 

7-1 

(a) Conversely, suppose there exist tQ > 0, p G (0, 1], and L slowly varying 
at infinity such that (5.2) holds fort = to- Then, there exists a function 
\{t) — >■ 0, implicitly defined by (5.16), such that (5.1) holds with a{t) = 
{ft, 1)~^ and v = F^^p. 

Remark 5.2. Note that if (6.1) holds for a weak solution v. [0, 00) — )• 
A4p with initial data vq, then (5.2) holds for t = 0. Similarly, if (5.2) holds 
for t = to = 0, then the converse result holds (cf. [16]). Indeed, the proof 
below extends easily to these cases. 



We begin our analysis leading to the proof of Theorem 5.1 with the fol- 
lowing pair of useful lemmas. 
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Lemma 5.3. (Uniform Convergence Lemma) Assume f > is monotone 
and regularly varying at x = oo with index p ^ 0. Assume h > 0. Then, for 
any ^ A ^ oo, 

(5.6) r 

as X ^ oo if and only if h{x) — )• A as x ^ oo. 



Proof. The result essentially follows from the Uniform Convergence 
Theorem of Karamata (see, for instance, [4, Theorem 1.5.2]). In particular, 
if / satisfies the hypotheses above, then the convergence f{\x)/f{x) — > A'' 
as X — )■ oo is uniform in A on compact subsets of E. Therefore, if h{x) — )■ A 
as x ^ oo for < A < oo, then (5.6) holds. The cases A = and A = oo 
then follow from the monotonicity of /. 

Conversely, suppose (5.6) holds for some ^ A ^ oo. Then, if h{x) A, 
there exists a subsequence 2;„ — ?> oo such that h{xn) — ?> z/ for some ^ ^ oo 
with V ^ \. We deduce that f{xnh{xn))/fixn) — > i^'', which contradicts 

(5.6) . This completes the proof. □ 

Lemma 5.4. Assume ^ : E ^ E is continuous and regularly varying at 
u = with index 7 > 1. Further, assume u : E ^ E solves the ordinary 
differential equation 

(5.7) u' = -^(u). 

Then u is regularly varying at infinity with index (1 — 7)"^. 



Proof. First assume u is invertible. Then it suffices to show that the 
function : (0,ti(0^)) — )■ -E is regularly varying at s = u(oo) = with 
index 1/(1 — 7)^""^ = 1 — 7. In that case, we apply Lemma 5.3 to the identity 

/ ^y-^ _ u-\u{tx)) _ 

J ~ u-^u{t)) ~ n-i(M(t)) ' 

to obtain 

u(tx) _j_ 
lim — -— = xi-T , 
t-s^oo u{t) 

for all X > 0. Hence, u is regularly varying at infinity with index (1 — 7)"^. 
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Obviously, u is decreasing when n > 0. Therefore, to show that u is 
invertible, we must show that u does not vanish in finite time. Writing (5.7) 
in integral form, we have 

(5.8) t-to= / T^^^- 

Ju{t) "^{w) 

Thus, u vanishes in finite time if and only if 'qj^dw < oo. Note that 
^(s) = s'^L(s) where L is slowly varying. Also, 



1 



(5.9) / — — -dw = -TTT I ^ \ dw = , ^ , , / ^ / \ dw. 

Since ^ is regularly varying at zero, the integral term on the right-hand 
side is bounded away from zero for s sufficiently small. Also, s^L(s) ^ as 
s ^ for all r > (see, for instance, [8, VIII.8]). Hence the left-hand side 
of (5.9) diverges as s — > 0, and we conclude that u is invertible. 

It remains to show that is regularly varying at zero with index 1 — 7. 
We consider any fixed < sq < ^(0"^). By a change of variables, (5.8) implies 

u-Hs) = u-Hso)-l^^^dw 

for all < s < u(0"'"). Therefore, by L'Hopital's rule, we obtain for any s > 

—s 

hm = hm ti-lX^ = hm ^ = hm £^ = 

*(t) 

This completes the proof. □ 

Solutions of the autonomous equation (5.7) have a translation invariance 
which plays an important role in our analysis. Specifically, if ^' > is con- 
tinuous and ^'(0"'') = (for instance, any critical branching mechanism), 
and if u ^ V > are solutions of (5.7) defined on E, then 

V{t) =u(t-T + U~'^{v{T))) 

for all t,T > 0. Recall that \i v : E ^ Aip is a weak solution of (1.11), 
then the function ip{-, q), defined by (3.4), solves (5.7) for all ^ g ^ cx). In 
particular, the function 

(5.10) rj{t) = {i^t A) = ^{t, 00) 
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solves (5.7). Since ip{t, oo) ^ <f{t,q) > , we obtain the identity 
(5.11) ^{t,q)=r,{t-T + ri-\^{r,q))) 

for all g > 0. Thanks to this identity, the characterization of scaling limits 
in the case of regularly varying branching mechanisms is relatively straight- 
forward. 

Proof of Theorem 5.1. Let and t] be defined by (3.4) and (5.10), 
respectively. Assuming (5.1) holds, we have 

ait)rj{t) = {ait)ut{X{tyUx), 1> ^ (i), l) 

Moreover, taking into account (5.11), we have 

(z>, 1 - e^«^-) = lim (a{t)i^tiXit)-^ dx), 1 - e'^^) 

t~^oo 

= lim a{t)ip{t,qX{t)) 

t— )-oo 



(5.12) =(z>, 1) lim 



l + ^V^\^{T,q\m 



1 

1-7 



where the last equality follows from Lemmas 5.3 and 5.4. Since the left-hand 
side is finite and independent of r, we conclude that there exists x{q) < ^ 
such that 

^V~'{v{r,q\it)))^x{q), 

for all r > 0. Since ^' > and ^(O"*") = 0, the function r], which solves (5.7), 
is decreasing and 7?(oo) = 0. Also, by the analysis of Section 3.2, (^(r, •) is 
increasing with ip{T,0) = for all r > 0. Hence, r/~^((^(T, •)) is decreasing 
with r/~^((/9(r, 0)) = +oo. Since the limit in (5.12) is non-constant in q, we 
must have xiQ) > and X{t) — )■ (otherwise, x vanishes on an unbounded 
interval). Therefore, 

\rj-\^{T,qXm .x(g)^Q^ 



ir?-i(^(r,A(t))) X(l) 

as t ^ oo. A standard rigidity lemma (see, for instance, [8, VIILS]) implies 
%('?) = x(l)'?'' foi^ some /5 ^ 0, and implies r]~^{ip{T, •)) is regularly varying at 
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q = with index p. Note that p < since x is non-constant. Also, since r/~^ 
is regularly varying at g = with index 1 — 7 (see the proof of Lemma 5.4), 
it follows that ^fir,-) is regularly varying at q = with index p = /5/(l — 7) 
for all r > 0. Therefore, 



(5.13) 0iq) (z), 1 - e-"-) = {O, 1> 1 + x{l)q 



■,p(l-7) 



1-7 



As a pointwise limit of Bernstein functions, is a Bernstein function. Hence, 
we must have < p ^ 1, otherwise if>" takes positive values near q = 0. 
Now let us show that (5.2) holds. For t > 0, we write 

/■oo 

cp{t, q)= (1 - e-''-)i/t(dx) ~ qPL{t, q~^) as <? ^ 0, 
Jo 

where L(t, •) is slowly varying at infinity. Next, we claim that qdqip{t,q) ~ 
p^{t, q) as q for all t > 0. Indeed, since Lfqg ^ 0, we have 

qdqf{t,q) l^jj;^ ^ 



(pit,q) x-1 
for all x > 1. Hence, 

hminf^^^^^liminf.^ 



g->o <p{t,q) <?^o X — 1 x — 1 

Also, the reverse inequality holds if we consider x < 1 and take the limit 
supremum instead. Thus, as x — >■ 1, we recover the limit qdqip{t, q)/ip{t, q) 
P- 

Therefore, we have 

poo 

dgip{t,q)= / e~'^^xi^tidx) pqP~^L{t,q~^) as q ^ 0. 
Jo 

This establishes a regular variation condition on the Laplace transform of 
the measure xvt{dx). By a classical Tauberian result (see, for instance, [16, 
Theorem 3.2]) we obtain the following equivalent condition on the distribu- 
tion function: 

I yviidy) ~ x^^^L(t,x) ■ — as x — >■ 00. 

Jo r(2-p) 

Hence, redefining L by a multiplicative factor, we obtain (5.2). 
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Finally, let us verify (5.4). A slight variation of estimate (5.12) gives, for 
all ^ g ^ oo. 



lim (a(s)z/sf(A(s) ^ dx),l 



-qx\ 



lim a{s)ip{st, qX{s)) 



{0, 1) lim 



t+-r,-\ip{T,qX{s))) 



1 

1-7 



1 

1-7 



(5.14) = {^,^'^'Pic^Ux), 

where fi^''^'^ is defined by (4.8), with /? = (7 — 1)^^(1^, 1)^"''^ and 



p(l-7)' 



1-7 



(5.15) 



d_ 

dq 



q=0 



chosen according with (4.11). The last equality, which is by no means obvi- 
ous, follows from (5.13). In particular, when p = 1, we obtain the relation 
cx = (p'{0) = {xu, 1) (cf. Theorem 6.1). 

Since (5.14) is valid for all ^ (7 ^ 00 (note, carefully, that we include 
q = 00) the continuity theorem (see, for instance, [8, XIII. 1]) implies that 
Ci{s)ust{X{s)~^ dx) converges vaguely to p^'^''^{c^^ dx). Also, the case q = 00 
implies convergence in total measure. We therefore obtain convergence in 
the weak topology (see, for instance, [1, Theorem 30.8]). Hence, taking into 
account (4.8), we obtain (5.3)-(5.5). This completes the proof of part (i) of 
the theorem. 

Now suppose there exists to > 0, p € (0,1], and L slowly varying at 
infinity such that (5.2) holds for t = Iq. Again, by the Tauberian theorem, 
we have dq(p{to, ■) regularly varying at g = with index p — 1. The regular 
variation of ^p{to, •) at = with index p then follows from the observation: 



(p{tQ,q) _ ip{to,q) - ip{to,0) _ dgip{to,qz) 



qdq(f{to,q) 



qdgip{to,, 



dg(f{to,q) 



dz 



1 



as g — 7- 0. 



The convergence of the integral term is easy to verify, see for instance [18, 
Lemma 3.3]. 
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Finally, for s > 0, let a{s) = t]{s)~^ and define X{s) by the relation 

(5.16) V'Mio,A(s))) = l. 

s 

It follows that 

s 

and we conclude, as above, that 



lim {rj{s) ^i/stiXis) ^ dx), 1 



t + q 



p(l-7) 



1 

1-7 



for all t > and for all ^ (7 ^ 00, where /3 = (7 — 1) ^. Weak convergence 
of the measures follows as before. This completes the proof. □ 

6. Scaling limits for fundamental solutions. In this section we 
show that a necessary condition for asymptotic self-similarity of fundamen- 
tal solutions is regular variation of the branching mechanism ^. In view 
of Definition (3.4) and property (ii) of Theorem (3.7), we consider critical 
branching mechanisms ^ for which Condition E holds. Our main result is 
the following: 

Theorem 6.1. Let fx: E Mp be the fundamental solution of (1.11), 
where is a critical branching mechanism verifying Condition E. Further, 
assume there exists a nonzero probability measure fi G Aip and a function 
A > such that 

I . fit{\{t)-^ dx) w 

1) 

as t — 7- 00. Then ^ is regularly varying at u = with index 7 G (1)2]. 
Furthermore, xjl € Mp and X{t)/{ij,t, 1) — )■ (x/t, 1) as t ^ 00. Moreover, we 
have the representation 

(6.2) fi = Fj^i{{xfi,iy^ dx) 

where F^^i is the generalized Mittag-Leffler distribution defined by (4.9). 

Before proving Theorem 6.1, we discuss a few basic properties of funda- 
mental solutions. Let us define, as before, the total measure function 

r]{t) = (//t,l) = ^{t,oo), 
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where $ is given by (3.7). Since •) is increasing, we have 



hm r](t) > hm (?) = q 

for ah q > 0. Hence, r/(t) — > cx) as t — >• 0. Moreover, r/ solves (5.7), where ^ > 
and ^(0^) = 0. Hence, r]{t) decreases to zero as t ^ oo. It fohows that 
7] : E E is bijective, and it is straightforward to check that its inverse is 
given by 



(6.3) 



1 



du. 



With this notation, $ in (3.7) has the representation 

(6.4) $(t,g) =7?(t + C(g)), 

which is a special case of (5.11). From this it follows easily that $ satisfies 
the forward equation 

(6.5) dt^ + ^{q)dq^ = 0. 
Finally, we note the following useful estimates. 

Lemma 6.2. Assume ^ is a critical branching mechanism verifying Con- 
dition E, and assume ( is defined by (6.3). Then the following hold for all 
s > 0; 



(n) 



ds^ 
d 



d 

^ ^ — 
ds 



> 0, 



(in) —[sC{s)]^Q. 

Proof. Part (i) is equivalent to the estimate 
(6.6) 



^is) , 2-^(s) 



s s 
The first inequality in (6.6) follows from the convexity of ^. That is, 

^(s) _ ^-(5) - $(0) 



s-0 
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Similarly, the concavity of ^' gives the estimate ^"{s) ^ ^'{s)/s, which 
implies 

/•s rs 

Jo Jo 
For the proof of (ii) , we compute 



1 



2^'{sf - *"(s)*(s) 



which is nonnegative by the estimate 



Finally, for the proof of (iii) , observe that 



(6.7) 

— fsC(s)l = / — r-r du -— = / — - — - du — / — • — 



For u ^ 1, part (i) implies 



Hence (6.7) is nonnegative, and the proof is complete. □ 

Proof of Theorem 6.1. Assuming (6.1), we have for all < g ^ 00 
that 

v{t + ciqxm /Him-' dx)^ ^ _ ^ ^^^^ ^ , ^ ^ _ 



V{t) \ {l-k, 1) 

as i — 7> 00. Equivalently, in the variable r == ?/(t), we have 

.^ex ~i ^ dcf ??(C(^) +C(gA(T))) 

(6.8) Vij.q) = > (p{q) 

T 

as r — 7- 0, where, for simplicity of notation, A(r) represents A(C(t)). Note 
that (p: [0,00) — )■ [0,00) is increasing from to 1, since /i is a probability 
measure. Also, note that ip is implicitly determined by the relation 

(6.9) C(9A(t)) = C(t(^(t,9))-C(t), 
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Claim: A(r) as t ^ 0. 
Proof of claim: Consider 

(6.10) ^(^) = ^- 

By part (i) of Lemma 6.2, F is nonincreasing, and r i-^ tF{t) is nondecreas- 
ing, lience 

F(tu) 1 ^ 1 F(tu) 

6.11 1^-^^-, foru^l, -^-^^1, forn>l. 

Therefore, equations (6.3) and (6.9) imply 

As r — > 0, the right-hand side is bomided away from zero for fixed g > 0. 
Also, F{t) oo since ^(0+) = and ^'(0+) = 0. It follows that C(gA(T)) 
oo as r ^> 0. Since C is decreasing on E and C(0'^) = oo, the claim follows. 

We now consider the rescaled equation 

(6.12) Cs('7A,(t)) = 6(T^(sr,g)) - C(t), 
where 

(6.13) Csir) = \s{t) = ——. 



We will show that equation (6.12) has a nontrivial limit 

(6.14) CiqXir)) = ((r^iq)) - C(r) 
as s — > 0, where 

(6.15) Xsir) ^ A(r) = (^'(0)r, (sir) ^ C(r) = r-^ 

for some r G (0, 1]. We will then show that the previous limits imply that ^ 
is regularly varying at zero with index 7 = r + 1 and that has a generalized 
Mittag-Leffier form determined by 



(6.16) ^(q) 



1 + {0'{O)q)- 
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The main idea is to show that subsequential hmits of (6.12) exist and are 
unique. We divide the proof into three main steps. 

Step 1. (Existence of subsequential limits.) 
First, we write 

(6,17, Ur) -l^r Ciu) . /' £gf . Z<i£) /' m 

Jl Jl (is) C(S) Jt "^{su) 

Note that for fixed s > 0, the function 

is increasing and convex. Furthermore, by part (i) of Lemma 6.2, we have 

(6.18) ^ ^'^(m) s^u for u < 1, n 'l's(n) ^ u^ for n > 1. 
On the other hand, by part (iii) of Lemma 6.2, for ah s > we have 

(6.19) ^(,)4.fZ<Me(0,l]. 

C{s) 

Claim: hmsup ^(s) > 0. 

Proof of claim: Assume for the sake of contradiction that ^(s) — ?■ as s — t- 0. 
Then, by (6.17) and (6.18), C is slowly varying at n = 0. By Helly's selection 
theorem, there exists a sequence Tj and a function 1 ^ f{u) ^ 1/n such 
that 

pointwise for u G (0, 1). Since is slowly varying and A(Tj) 0, we have 

^{q) U j^oo F{Tj) j^oo C(A(r,-)) F{Tj) j-^oo F{Tj) 

This gives a contradiction since if is nonconstant and the right-hand side is 
independent of q. Therefore, the claim holds. 

Now we may apply Helly's selection theorem to find a sequence 5^—7-0 
and a function > for which 
(6.20) 

Cisk) I for some ^ G (0, 1], ^s^iu) ^(u) for ah u > 0. 
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Furthermore, as a pointwise limit of convex functions, ^ is convex. By dom- 
inated convergence, 

(6.21) Cs,(t) ^ C(r) = 1 + I /' 7^ du. 

Since ^ is convex and positive, C G C^{E) and is strictly decreasing, and the 
convergence in (6.21) occurs locally uniformly for t £ E. Now by assumption 
(6.8), the right-hand side of (6.12) converges to 

(6.22) RiT,q)'^CiMq))-Cir)>0 

for all r, (/ > 0. Hence, the left-hand side of (6.12) also converges, and if 
is defined on E, then 

(6.23) Xs,{T)^X{T)'='q-'c\R{r,q)). 
Claim: (: E ^ E is a bijection. 

Proof of claim: Recall is strictly decreasing. For r ^ 1, estimate (6.18) 
implies 

. /■! 1 

C(r) ^ 1 + ^ - du = 1 - ^ luT ^ oo asT^O. 

Jt U 

It remains to show that ("(r) — )• as r — )• oo. Assume, for the sake of 
contradiction, that C,{t) — > L > as r — > cxd. Since ip{q) — )■ 1 as q — )■ oo, we 
may choose for any r > a value g > sufficiently large so that i?(r, q) < L. 
It follows 

(6.24) '^Ski'^) — > oo as A; — )■ oo, 

for otherwise, along some bounded subsequence, the left-hand side of (6.12) 
would have a subsequential limit with value larger than L, which is a con- 
tradiction. But now (6.24) implies that for all g > 

R{T,q) = lim Cs,{qK{r))^h 

since Cs{u) ^ 1 for any s > and u ^ 1. However, by (6.22), R{T,q) — > oo 
as g — 7- 0, since ^ is unbounded above and ip ^ as q 0. This is a 
contradiction, which gives the claim. 
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We conclude that, along the sequence — 0, equation (6.12) has a well- 
defined limit of the form (6.14) for all T,q > 0. Furthermore, by (6.12) and 
(6.17), we have 



^ 1 .... r 1 



In particular, fixing g > and taking into account (6.18) shows that 
(6.25) ^Ski'T) ^{'t) locally uniformly for t ^ E. 

This fact will play a role in the uniqueness proof to follow. 

Step 2. [Uniqueness of sub sequential limits.) 

We now show that subsequential limits obtained as in Step 1 are unique. 
First, equations (6.14) and (6.21) imply 



(6.26) C{r) = U F{s)- = d .^F{qs)-, F{u) 



dcf U 



It follows that 



T^(q) s J^iM s ^(n) 



(f [q) ^ ^ tor all T,q > U. 



Therefore, (p'{q) = f^e '^^xfi{dx) is decreasing and bounded above, and we 
deduce that 

0<(^[,^='<^'(0+) <oo. 
Furthermore, taking g ^ in (6.26) gives 

(6.27) C(r)=iFo\n^, with Fo = ^(0+). 

Formally, we have Fq = oo ii and only if A(r) = ip'^r for all r > 0. More 
precisely, note that the left-hand side of (6.26) is positive, so that F{qs) has 
a finite limit as — )• if and only if A(t) > c^qT for each r > 0. 
On the other hand, equations (6.14) and (6.21) also imply 



.28) C(gA(r)) = e/ F{s)-=d F{ts) -. 
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Taking r — t- implies 

(6.29) ^(qX,) = iFoln^, with Ao = A(0+). 

In particular, Aq = if and only if Fq = oo. Note also Aq < oo, since Fq ^ 1. 
We now consider two cases. 

Case 1: (Aq = 0). As noted above, Aq = if and only if A(t) = (PqT for all 
T > 0. Hence, (6.14) reduces to 

(6.30) C(Tg^o) = C(r(^(9))-C(T). 
Differentiating in q and r gives the relations 

(6.31) (^(g) _ Fjrm) _ ^ , Hr) 



Therefore, F{Tq(p'Q) / F{t) is constant in r and we deduce that F is a power 
law: F{u) = , since F{1) = 1. Note that r 7^ 0, since Fq = 00. It then 
follows from (6.18) and (6.26) that < r ^ 1. The second equality above 
reduces to 

(6-32) |^ = l + (,^^)^ 

which gives (6.16). On the other hand, '^{u) = u^~^^, so that (6.21) implies 

o = c(oo) = i-e/ —^du = i-^. 

Hence, ^ = r. In summary, we obtain in this case: 

(6.33) X{r)=^or, 4'{t)=t'+\ C(r) = T-^ | = r, 
where < r ^ 1 and (p is given by (6.16). 

Case 2: (Aq > 0). We will show that the remaining case, Aq > 0, leads to a 
contradiction. We divide this case into three parts. 

(i) First, let us show that if Aq > 0, then has the form (6.16), and 

(6.34) A(r)'^ = AS + (r^'o)^ 
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The idea is to consider a rescaling of (6.14), of the same form as (6.12); 
namely, 



(6.35) 
where 

(6.36) 
Since C{t) 
(6.37) 



UqUr)) = UMq)) - Lir) 



Cs(r) 



drf Cjsr) 
~7(^' 



As(r) 



XisT) 



as T ^ 0, we deduce from (6.27) that 
X{st] 



ST 



as s — > oo. 



Furthermore, since Lemma 6.2 applies to the functions ^ and C, we can use 
Kelly's selection principle, as in Step 1, to pass to the limit in (6.35) along 
some sequence Sfc — ?> oo. Up to relabeling, the limit equation matches exactly 
the form (6.30). In particular, (6.37) implies that Qk ^ nonconstant 
limit, and we obtain, as before, (6.16) from the relations (6.31). Note that 
the constant r in (6.16) is the same as in the previous case, since is fixed. 

Now, substituting q = t/Xq in (6.29) and comparing with (6.27), we 
obtain 



(6.38) 



A(r) 



1 



for all r > 0. Using (6.16) in the previous relation gives (6.34). In particular. 



(6.39) 



A(r) 



is decreasing function of r > 0. 

(ii) Next, let us show that if (6.39) holds, then 



(6.40) 



lim sup 



A(r) 



oo 



and 



lim inf 



A(r) 



Recall that A(r) = limfc_>,oo -^Sfc(''") some Sk — > 0. Therefore, (6.39) implies 

A(sfcr) A(r) 



lim 

A;— >oo SfcT 



OO 
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as T — >■ 0, and the first statement in (6.40) follows. 
On the other hand, by (6.9) 

^>^, forallr,g>0. 
r q 

Hence, for all t > and for all (7 > 0, 

t k^oo Skt T->0 r T-^o q q 

Taking into account (6.39) and passing to the limit i — > 00 on the left and 
(7 — 7> on the right yields the last statement in (6.40). 

(iii) Finally, we show that (6.39) and (6.40) lead to a contradiction. Fix M > 
m > (p'q, and choose a sequence of disjoint intervals [a^, bk] as follows. 

1. Choose bk^O such that > M. 

bk 

2. Define Ck = sup{r < bk '■ ^ ^ < m}. 

T 

3. Choose Ofc such that 1 < — < 1 + — and ^ ^'^ < m. 

ak k Qk 

Taking s = ak and r = 1 in (6.12), we have 

(6.41) Ca,iqK,{l))=Ca,mak,q))-l. 
Since 

. v . rHok) ^ A(afc) 
ipQ ^ lim mf ^ lim sup ^ m 

ak k~>oo Clk 

and ip{ak,q) — 0iq) as k 00, it follows that the sequence (,{ak), defined 
by (6.19), is bounded away from zero; otherwise, there exists a subsequence 
CakX"^) ~^ 1' which contradicts (6.41). Therefore, as in Step 1, (6.12) has a 
nontrivial limit along a subsequence a^^ — t- 0, j ^ 1. In particular, the local 
uniform convergence of A^^ implies 

•^afeW A(r) 

(6.42) > — ^ locally uniformly for t e E, 

T T 



where A satisfies (6.34), or, equivalently, 
(6.43) 



1 

Mr) Mi , /-/Nr-' 
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If Aq > 0, then (6.43) is strictly decreasing in r, and we have, for all t > 1, 

^ < A(l) = lim Xa,il)^m. 

On the other hand, if Aq = 0, then A(r)/r = A(l) = ip'^ < m for ah r > 0. 
Hence, in either case, we have A(r)/r < m is non-increasing for all r > 1. 

Next, choose e < min{M — m, m — — ^}, and choose J large enough so 
that 



(^) A(t) 



(6.44) 

Since rj = b^. / a^^ > 1 and 



<e Vj^J, VtG[1,3]. 



A(^) Mr,) 



> \M - m\ > e, 



it follows from (6.44) that rj > 3 for all j ^ J. Therefore, 
Kir) 



(6.45) 



> m 



yj^J, Vr e [2,3] C (cfc,/afc,, r 



since, by construction, \{t)/t ^ m for all r G (cfc,6fc]. Hence, (6.45) implies 



(6.46) 



K^ {r) A(r) 



m 



Mr) 



m 



A(2) 



> e 



for all j ^ J and for all r S [2,3]. This contradicts (6.44). Therefore, the 
hypothesis of Case 2, Aq > 0, is never satisfied, and we obtain in Step 1 
unique subsequential limits of the form (6.33). 

Step 3. {Limit as s ^ 0.) 

To finish the proof of the Theorem, note that we must have ^{s) — )• = r 
as s — > 0. Otherwise, by Step 1, it is possible to extract subsequential limits 
with distinct values of ^, contradicting (6.33). Similarly, the full limit of 
each of the rescaled functions Q, ^s, and As exists as s — > 0, since given 
any sequence Sk — > 0, there exist unique subsequential limits by Steps 1 
and 2. In particular, (6.33) shows that ^ is regularly varying with index 
7 = r + 1 € (1, 2]. Also, (6.32) implies 

1 



[1 + {^'{Q)q)~ 



nl/r' 
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where (p'{0) = {xfi, 1). This gives (6.2). Finally, (6.15) implies that 
X{C{st))_ X{C{st)) 



r/(C(sr)) 



T 



A,(r) ^(^'(O)r 



where we recall that the term A(t), first appearing in (6.9), is shorthand 
notation for A((^(r)), where A > is the scaling function in (6.1). It follows 
that X{t) ~ ip'{0)r]{t) = {x/l, 1) (/i^, 1) as t — >■ oo, and the proof is complete. 

□ 

Remark 6.3. The conclusions of the Theorem follow much more quickly 
if one assumes that the scaling function X{t) ~ (Aii,l) in (6.1), based on 
the arguments of Pakes [20] which make use of the forward equation (6.5). 
Testing (6.1) with xe~'^^ it follows 



(6.47) d,<^it,Xq) = ^^^^^0'iq), q > 0. 



Writing u = Xq and noting 9 = 0{q)/q is a monotonic function of q, we have 
that $(t, Ag) = u6'(l + o(l)) and thus 

(6.48) ^M(l + o(l))) ^ 

W(ti) 

as li ^ 0. By simple estimates based on the continuity and monotonicity of 
^, one can eliminate the 1 + o(l) factor and conclude that ^ is regularly 
varying by the standard rigidity lemma in [8, VIII.8]. 

7. Limit theorems for critical CSBPs. We conclude this paper by 
applying the results in Sections 5 and 6 to derive limit theorems for critical 
CSBPs that become extinct almost surely. First, we obtain a conditional 
limit theorem for fixed initial population x. In particular this solves the 
continuous-state analog of the open question posed by Pakes in [20, Remark 
6.1]. 

Theorem 7.1. Assume Z(t,x) is a continuous- state branching process 
with critical branching mechanism ^ verifying Condition E. Further, assume 
that for some (equivalently all) x > 0, there exists a function A > and a 
probability measure (1 £ A4f such that 

(7.1) F{X{t)Z{t,x) z \ Z{t,x) > 0) ^ I fL{dz) 
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vaguely as distribution functions. Then, there exists 1 < 7 ^ 2 such that 
^ is regularly varying at u = with index 7. Furthermore, x(l G Mp and 
X{t) ~ {xfL, l)¥{Z{t, x) > 0) ast^ 00. 

Conversely, suppose ^ is regularly varying at u = with index 1 < 7 ^ 2. 
Then, (7.1) holds with X{t) = F{Z{t,x) > 0) and p, = F^^i{dz). 



Proof. It follows from (1.2) that 



(7.2) F{Z{t,x) = 0) = lim E(e-«^(*'^)) = e-^^(*'~) = e-^^'^*'^^ 

with fit the Levy measure for Z(t,x). By the continuity theorem [8, XIII. 1], 
(7.1) implies 



[ e-'^yfi{dy) = lim 
Je 



]E(g-gA(t)Z(M)) _ f>[Z{t,x) = 0) 



i^oo F{Z{t,x)>0) 



= lim , 

t-s.oo 1 — e-^\A"'i> 

Hence, 

1 _ g-^'p{t,Ht)q) 



lim — — = / (1 - e-'iy)fi{dy), 



where the second equality follows by Taylor expansion and the fact that 
< ip{t,X{t)q) < (/i(t),l) — > as f — >■ 00. Since /it is the fundamental 
solution of the associated equation (1.11), we conclude, by Theorem 6.1, 
that there exists 1 < 7 ^ 2 such that ^ is regularly varying at u = with 
index 7. Also, by Theorem 6.1, 

A(t) ~ {xfi, l){f,t, 1) ~ {xfi, 1)(1 - e-^/^*'!)) = {xfi, l)nZ{t, 1) > 0) 

as t —7- 00. The converse follows easily from Theorem 5.1. This completes 
the proof. □ 

Next, based on the same results on scaling limits of fundamental solutions, 
we study scaling limits as t — )■ 00 of CSBPs with initial population scaled to 
obtain non-degenerate Levy process limits x 1— > Z(x). As in [10, Ch. VI], let 
D to denote the space of cadlag paths equipped with the Skorokhod topology. 
We use the notation = to denote equality in law (i.e. both processes define 
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the same measure on the Skorokhod space D), and the notation A- to denote 
convergence in law for these processes (i.e. weak convergence of the induced 
distributions on the Skorokhod space). 

For convenience, we introduce a notation for rescaled processes. If A, a > 
0, and X ^ X{x) is a process, then we define the rescaled process 5\^aX by 
^\,aX{x) '= XX{ax). 

Theorem 7.2. Let Z{t,x) be a continuous state branching process with 
critical branching mechanism ^ verifying Condition E. 

(i) Assume there exists a Levy process Z = Z{x) and functions a, A > 
such that 

(7.3) <5;,(i),„(i)Z(t,.)-^Z(.). 

^ ' ^ ' t— 5.00 

Further, assume the non- degeneracy condition 

(7.4) lim P(Z(t, a{t)x) = 0) = P(i'(x) = 0) € (0, 1) 

for some x > 0. Then, there exists 1 < 7 ^ 2 such that ^ is regularly 
varying at u = with index 7, and there exist constants Ca,c\ > 
such that 

(7.5) ^ ~ ^ ~P(Z(t,l) > 0) as t^ 00. 
a{t) cx 

Furthermore, for all (fixed) t > 

(7.6) Sx(s),ais)Z{st,-) 6tj*j--^*Z, where 7*=^^—. 

^ ' ^ ' s— >-oo ' 7 — i 

Also, for all t > 0, 

(7.7) (5l,c,c;,^/3,7(t, •) = ^a* ,t--f' ^ 

where Zp^^{t,-) is the continuous-state branching process with branch- 
ing mechanism ^(n) = /3u'^ with (3 = ^*ci^'^ . 

(ii) Conversely, assume ^ is regularly varying at zero with index 1 < 7 ^ 2. 
Then (7.6) holds with \{s) = a{s)~^ = P(Z(s, 1) > 0), where Z{x) is 
the Levy process with Levy measure F^^i[dx) defined by (4.9). 

Proof. Since we are dealing with increasing Levy processes, the process 
convergence in (7.3) is equivalent to the pointwise convergence of Laplace 
exponents 

(7.8) a{t)ip{t, X{t)q) > ip{q), for ah q G [0, 00), 



42 G. IYER, N. LEGER AND R. L. PEGO 

where x(p{q) = — lnE(e~^'^^^'^) is the Laplace exponent of Z{x). (See, for in- 
stance [10, XV. 3. 2, XIII. 1.2] or the proof of Theorem 1 in [17].) Furthermore, 
by (7.4) we must have 



hm hm xa{t)(p{t, X(t)q) = hm hm — hiE(e' 



= Hm -lnF{X{t)Z{t,a{t)x) = 0) = -lnF{Z{x) = 0) = Hm xip{q), 
and hence 

(7.9) Hm a(t)v?(t, oo) = (p{oo) G (0, oo). 

t— >-oo 

Then, denoting the Levy measures of Z(i, x) and Z{x) by jit and /i, respec- 
tively, we deduce from (7.8) and (7.9) that 

(7.10) ^_^,(A(t)-idx) ^ j^Kdx). 

Therefore, by Theorem 6.1, there exists 1 < 7 ^ 2 such that is regularly 
varying at u = with index 7. Moreover, 

A«) ~ ^(M,. 1) ~ ^(1 - = ^P(Z(*. 1) > 0), 

Hence, together with (7.9), we obtain (7.5) with = (/i, 1) and c\ = ^j^^- 
Also, by Theorem 6.1, 

/i(dx) = CaF^^i{c~^dx) = CaCx [c^^F^,i (c^^rfx)] = CaCxfJ.i''^'^{dx) 



where fi^^'"''^ is defined by (4.8) with f3 = ^4 • Therefore, by Theorem 5.1, 
we have 



(7.11) aisMst, Xis)q) ^ f-^' 0it^* q) = c^cx V^'^'\t, q) 

as s ^ 00 for all ^ g ^ 00, where y?^'"^'^ is defined by (4.10). Since 

we obtain (7.6). Similarly, we obtain (7.7) from (7.11). 

For the converse, we observe that the convergence in (7.3) holds if and only 
if the Laplace exponent converge pointwise as in (7.8) (see, for instance [10, 
XV. 3. 2, XIII. 1.2] or the proof of Theorem 1 in [17]). The converse now follows 
by a similar argument. □ 
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Remark 7.3. The non-degeneracy condition (7.4) has the following in- 
terpretation. The spatial process x i— )• Z^''^{x) is a compound Poisson process 
with jump measure a{t) ^t{^{t)~^ dx) and scaled intensity a{t){fit,^)- One 
thing that (7.4) means is that we assume the scaled intensity converges to 
the intensity of jumps {fi, 1) = 1 in the limiting process Z. In particular this 
presumes there are no small jumps with finite intensity being lost in the 
limit. 
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